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Servicemembers 


Why  GAO  Did  This  Study 

DOD  reimburses  the  Department  of 
Labor  quarterly  for  unemployment 
compensation  payments  provided  to 
former  servicemembers.  In  2015,  DOD 
reported  that  unemployment 
compensation  for  veterans  was  $434 
million.  Differences  between  military 
and  civilian  occupational  classification 
systems  can  make  it  difficult  for 
servicemembers  to  identify  civilian  jobs 
that  are  comparable  to  their  military 
occupational  specialties.  Section  2015 
of  Title  10  directed  DOD  to  carryout  a 
program  to  enable  servicemembers  to 
obtain  professional  credentials  related 
to  their  military  training  that  translate 
into  civilian  occupations. 

Senate  Report  114-49  accompanying 
the  National  Defense  Authorization  Act 
for  Fiscal  Year  2016  included  a 
provision  for  GAO  to  review  DOD’s 
credentialing  program.  This  report  (1) 
assesses  the  extent  to  which  DOD 
implemented  the  statutorily  required 
credentialing  program  and  developed 
performance  measures  to  guide  the 
program;  and  (2)  describes  how  DOD 
engages  with  states  to  enhance 
opportunities  for  servicemembers  to 
attain  credentials.  GAO  reviewed  DOD 
policies  and  procedures  and  other 
documentation  and  interviewed 
relevant  officials. 

What  GAO  Recommends 

GAO  recommends  that  DOD  develop 
and  implement  performance  measures 
for  its  credentialing  program.  DOD  did 
not  concur  with  the  recommendation, 
but  GAO  continues  to  believe  that 
performance  measures  are  needed  to 
measure  the  success  of  the  program, 
as  discussed  in  the  report. 


View  GAO-17-133.  For  more  information, 
contact  Brenda  S.  Farrell  at  (202)  512-3604  or 
farrellb@gao.gov. 


What  GAO  Found 

The  Department  of  Defense  (DOD)  has  taken  steps  to  establish  the  statutorily 
required  credentialing  program,  but  it  has  not  developed  performance  measures 
to  gauge  the  program’s  effectiveness.  Office  of  the  Secretary  of  Defense  (OSD) 
officials  are  in  the  process  of  coordinating  a  draft  DOD  instruction  for  the 
program  that  will  assign  responsibilities  and  prescribe  procedures  for  its 
operation,  and  each  of  the  services  has  established  websites  to  help  its 
servicemembers  find  information  on  certifications  and  licenses  related  to  their 
jobs.  Servicemembers  can  use  these  websites  to  obtain  background  information 
on  credentialing  and  detailed  information  on  credentials  related  to  a  military 
occupational  specialty,  credential  requirements,  potential  gaps  between  military 
training  and  civilian  credentialing  requirements,  and  resources  available  to  fill  in 
those  gaps.  Neither  the  draft  DOD  instruction  nor  any  other  guidance  provided 
by  OSD,  however,  establishes  any  performance  measures  by  which  to  evaluate 
the  program’s  effectiveness.  By  using  performance  measures,  conducting 
ongoing  monitoring,  and  reporting  on  progress  in  meeting  its  desired  outcomes 
transparently,  key  decision  makers  can  obtain  feedback  for  improving  the 
program.  DOD  officials  collected  some  credentialing  data  for  fiscal  year  2015, 
such  as  the  number  of  credentials  that  active  duty  and  reserve  servicemembers 
successfully  attained  and  expenditures  for  the  credentialing  program.  However, 
without  performance  measures  that  have  targets  and  a  baseline  against  which  to 
measure  current  performance,  DOD  officials  and  other  decision  makers  may  find 
it  difficult  to  determine  whether  DOD's  credentialing  program  is  on  track  to 
achieve  desired  results  or,  alternatively,  needs  corrective  actions. 

DOD  engages  with  states  in  various  efforts  to  further  assist  servicemembers  in 
attaining  credentials,  including  working  at  the  state  level  to  encourage  and 
support  the  development  of  credentials  for  servicemembers,  with  efforts  varying 
by  state.  For  example,  DOD  created  the  “USA  4  Military  Families”  initiative  to 
engage  with  state-level  policymakers,  not-for-profit  associations,  concerned 
business  interests,  and  other  state  leaders  regarding  the  needs  of  military 
members  and  their  families.  DOD  also  provided  data  and  subject  matter 
expertise  to  an  18-month  partnership  project  with  the  Department  of  Labor  and 
National  Governors  Association.  This  project  was  designed  to  identify  state-level 
professional  requirements  that  can  be  met  through  the  training  received  by 
servicemembers  in  the  armed  forces  and  to  identify  strategies  to  remove  barriers 
to  servicemembers’  efforts  to  attain  credentials.  The  six  states  participating  in  the 
project  found  that  transitioning  servicemembers  and  veterans  encounter  various 
barriers  when  trying  to  attain  civilian  credentials,  such  as  civilian  licensing  boards 
that  are  not  accustomed  to  military  documentation  of  a  servicemember’s  training 
and  experience.  The  project  also  identified  strategies  to  enable  states  to 
accelerate  the  licensing  and  certification  of  veterans  based  on  the  challenges 
identified,  such  as  working  with  educational  institutions  to  set  up  accelerated 
programs  that  provide  veterans  advanced  standing  in  existing  programs,  or  offer 
bridge  courses  to  prepare  veterans  entering  existing  programs. 
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GAO 

441  G  St.  N.W. 
Washington,  DC  20548 


U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


October  17,  2016 

The  Honorable  John  McCain 
Chairman 

The  Honorable  Jack  Reed 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Mac  Thornberry 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

From  2000  through  2015,  Department  of  Defense  (DOD)  data  indicate 
that  unemployment  compensation  for  military  service  veterans  increased 
from  $170  million  to  a  peak  of  $944  million  in  201 1  before  declining  to 
$434  million  in  201 5. 1  According  to  DOD,  some  servicemembers 
experience  difficulty  finding  civilian  jobs  after  leaving  the  service,  even 
though  they  typically  receive  extensive,  high-quality  training  during  their 
military  careers,  as  well  as  work  experience,  that  can  prepare  them  for  a 
wide  range  of  occupational  specialties.  This  difficulty  is  increased  by  the 
fact  that  differences  between  military  and  civilian  occupational 
classification  systems  can  make  it  difficult  for  separating  or  retiring 
servicemembers  to  identify  jobs  in  the  civilian  workforce  that  are 
comparable  to  their  military  occupational  specialties. 

To  help  ensure  that  servicemembers’  skills  can  be  applied  to  meet  the 
needs  of  civilian  employers  when  they  leave  the  military,  the 


Sections  8521-25  of  Title  5  of  the  U.S.  Code  provide  fora  program  of  unemployment 
compensation  for  unemployed  individuals  separated  from  the  Armed  Forces.  DOD 
components  are  responsible  for  budgeting  unemployment  compensation  benefits  for 
unemployed  ex-servicemembers.  An  ex-servicemember  is  eligible  for  unemployment 
compensation  if  he  or  she  was  discharged  or  released  under  honorable  conditions  and 
has  completed  his  or  her  first  full  term  of  active  service,  or  meets  other  qualifying 
conditions.  5  U.S.C.  §  8521(a)(1)(A)-(B).  The  Department  of  Labor  is  the  executive  agent 
for  the  program,  but  administration  is  accomplished  by  each  state. 
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administration,  in  2012,  created  the  DOD  Military  Credentialing  and 
Licensing  Task  Force.  This  task  force  was  charged  with  identifying 
military  specialties  that  readily  transfer  to  high-demand  civilian  jobs, 
working  with  civilian  credentialing  associations  to  address  gaps  between 
military  training  programs  and  requirements  for  credentialing,  and 
providing  servicemembers  with  greater  access  to  necessary  certification 
and  licensing  exams.2  Congress  also  took  action  to  address  the 
credentialing  of  servicemembers  by  enacting  section  551  of  the  Carl 
Levin  and  Howard  P.  “Buck”  McKeon  National  Defense  Authorization  Act 
(NDAA)  for  Fiscal  Year  2015,  which  amended  10  U.S.C.  §  2015  to  require 
DOD  to  carry  out  a  program  to  enable  members  of  the  armed  forces  to 
obtain,  while  serving  in  the  armed  forces,  professional  credentials  related 
to  their  military  training  and  skills.3 

Senate  Report  1 14-49  accompanying  the  NDAA  for  Fiscal  Year  2016 
included  a  provision  for  us  to  review  DOD’s  credentialing  program.4  This 
report  (1)  assesses  the  extent  to  which  DOD  implemented  the  statutorily 
required  credentialing  program,  developed  performance  measures  to 
guide  the  program,  and  identified  any  challenges;  and  (2)  describes  how 
DOD  engages  with  states  to  enhance  opportunities  for  servicemembers 
to  attain  credentials. 

For  our  first  objective,  we  interviewed  officials  in  DOD  and  each  of  the 
military  services  and  obtained  documentation  where  available  to 
determine  the  extent  to  which  the  credentialing  program  has  been 
implemented,  identify  any  challenges  associated  with  implementation, 


According  to  DOD,  credentialing  is  an  umbrella  term  to  cover  both  “certifications”  and 
“licenses.”  “Certifications”  refers  to  the  voluntary  process  by  which  a  nongovernmental 
entity  grants  a  time-limited  recognition  and  use  of  a  credential  to  an  individual  after 
verifying  that  he  or  she  has  met  predetermined  and  standardized  criteria.  “Licensure” 
refers  to  the  mandatory  process  by  which  a  governmental  agency  grants  time-limited 
permission  to  an  individual  to  engage  in  a  given  occupation  after  verifying  that  he  or  she 
has  met  predetermined  and  standardized  criteria. 

3Pub.  L.  No.  113-291,  §  551  (2014).  These  credentials  must  be  related  to  military  training 
and  skills  that  are  acquired  during  service  in  the  armed  forces  incident  to  the  performance 
of  the  servicemember’s  military  duties  and  translate  into  civilian  occupations.  Id. 

4Senate  Report  114-49  refers  to  the  program  as  a  “credentialing  and  licensing  programs.” 
DOD  refers  to  its  program  as  a  “credentialing  payment  program”  because  DOD  generally 
defines  “credentials”  as  including  both  certifications  and  licenses.  This  report  uses  DOD’s 
terminology  to  describe  the  program. 
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and  collect  any  data  and  methodologies  used  to  measure  the 
implementation  and  effectiveness  of  the  program.  We  also  obtained  and 
reviewed  fiscal  year  2015  credentialing  program  data  from  the  Army,  the 
Air  Force,  and  the  Navy.  We  assessed  the  reliability  of  this  information  by 
reviewing  the  program  data  and  service  officials’  responses  to  our  data 
reliability  questionnaire.  We  found  this  information  and  data  to  be 
sufficiently  reliable  for  the  purpose  of  reporting  cost  information  and  the 
numbers  of  credentials  obtained  by  the  services.  To  assess  how  DOD 
measures  the  effectiveness  of  its  credentialing  program,  we  used  federal 
Standards  for  Internal  Control5  to  review  DOD’s  draft  policy  to  determine 
whether  it  included  controls  for  monitoring  and  key  attributes  of 
performance  measures,  such  as  measurable  targets  and  baseline  data, 
outlined  in  our  previous  work.6 

For  our  second  objective,  we  interviewed  officials  at  DOD — including  the 
Defense  State  Liaison  Office  and  the  military  services — as  well  as  the 
Department  of  Labor  to  determine  how  DOD  engages  with  states  to  foster 
further  development  of  servicemembers’  credentialing  programs.  We 
obtained  and  reviewed  previous  reports  on  DOD’s  efforts  to  address 
servicemember  credentialing,  such  as  the  National  Governors 
Association’s  final  report  on  its  project  related  to  states’  experiences  with 
veterans’  credentialing  and  DOD’s  report  on  civilian  credentialing  for 
military  occupations.  We  also  obtained  and  reviewed  the  Executive  Office 
of  the  President’s  2012  and  2013  reports  on  leveraging  and  streamlining 


5See  GAO,  Standards  for  Internal  Control  in  the  Federal  Government,  GAO/AIMD-OO- 
21.3.1  (Washington,  D.C.:  Nov.  1999).  These  standards  were  in  effect  prior  to  fiscal  year 
2016  and  cover  the  time  period  of  government  agencies’ data.  The  standards  were 
subsequently  updated.  The  updated  standards  went  into  effect  on  October  1, 2015.  See 
GAO,  Standards  for  Internal  Control  in  the  Federal  Government,  GAO-1 4-704G 
(Washington,  D.C.:  Sept.  10,  2014). 

6See  GAO,  Tax  Administration:  IRS  Needs  to  Further  Refine  Its  Tax  Filing  Season 
Performance  Measures,  GAO-03-143  (Washington,  D.C.:  Nov.  22,  2002),  p.  45,  fora 
description  of  how  we  developed  the  attributes  of  effective  performance  goals  and 
measures.  We  also  reviewed  the  Government  Performance  and  Results  Act  of  1993.  See 
also  GPRA  Performance  Reports,  GAO/GGD-96-66R,  (Washington,  D.C.:  Feb.  14,  1996); 
Missile  Defense:  Opportunity  to  Refocus  on  Strengthening  Acquisition  Management, 
GAO-13-432  (Washington,  D.C.:  Apr.  26,  2013);  Performance  Measurement  and 
Evaluation:  Definitions  and  Relationships,  GAO-1 1-646SP  (Washington,  D.C.:  May  201 1 ); 
and  Agency  Performance  Plans:  Examples  of  Practices  That  Can  Improve  Usefulness  to 
Decisionmakers,  GAO/GGD/AIMD-99-69  (Washington,  D.C.:  Feb.  26,  1999). 
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credentialing  for  servicememebers,  veterans,  and  their  spouses.7  On  the 
basis  of  DOD  demographic  data  on  the  number  of  active  duty 
servicemembers  in  each  state,  we  selected  three  states  with  the  highest 
numbers  of  active  military  servicemembers  to  contact  for  our  examination 
of  DOD’s  efforts  to  work  with  state-level  officials.  While  information  we 
obtained  from  these  three  states  is  not  generalizable,  it  provides  valuable 
examples  of  initiatives  the  states  are  undertaking  on  behalf  of 
servicemembers.  We  interviewed  DOD  regional  liaisons,  state-level 
government  officials,  an  official  representing  a  state-level  non¬ 
government  organization,  and  state-level  legislative  liaisons  who  work 
with  DOD  on  state-  and  local-level  credentialing  efforts,  to  determine  the 
degree  to  which  they  collaborate. 

We  conducted  this  performance  audit  from  September  2015  to 
September  2016  in  accordance  with  generally  accepted  government 
auditing  standards.  Those  standards  require  that  we  plan  and  perform  the 
audit  to  obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable 
basis  for  our  findings  and  conclusions  based  on  our  audit  objectives.  We 
believe  that  the  evidence  obtained  provides  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives. 


Background 


According  to  officials  in  the  Office  of  the  Under  Secretary  of  Defense  for 
Personnel  and  Readiness,8  when  military  servicemembers  leave  active 
duty  they  may  have  trouble  finding  civilian  jobs.  A  veteran  entering 
private-sector  employment  for  the  first  time  may  lack  the  familiarity  with 
effective  job  search  strategies  that  someone  with  civilian  work  experience 
could  be  expected  to  have.  While  in  the  service,  veterans  may  have  held 
jobs  that  do  not  exist  in  the  civilian  world,  and  others  may  have  difficulty  in 


7See  National  Governors  Association  Center  for  Best  Practices,  Workforce  Development 
Program,  Veterans’  Licensing  and  Certification  Demonstration:  A  Summary  of  State 
Experiences ,  Preliminary  Findings,  and  Cost  Estimates  (Sept.  2015);  DOD,  Pilot  Program: 
Civilian  Credentialing  for  Military  Occupational  Specialties  (Sept.  27,  2013);  Executive 
Office  of  the  President,  Military  Skills  for  America’s  Future:  Leveraging  Military  Service 
and  Experience  to  Put  Veterans  and  Military  Spouses  Back  to  Work  (May  31 , 2012);  and 
Executive  Office  of  the  President,  The  Fast  Track  to  Civilian  Employment:  Streamlining 
Credentialing  and  Licensing  for  Service  Members ,  Veterans,  and  Their  Spouses  (Feb. 
2013). 

8The  Under  Secretary  of  Defense  for  Personnel  and  Readiness  is  responsible  for 
maintaining  DOD’s  credentialing  program  and  monitoring  implementation  and  compliance. 
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determining  how  to  adapt  the  skills  they  acquired  in  the  military  to  a 
different  context,  even  where  those  skills  are  more  directly  applicable. 
Numerous  actions,  such  as  the  following,  have  been  taken  to  help 
servicemembers  find  civilian  jobs  when  they  leave  the  military: 

•  Congress  passed  the  VOW  to  Hire  Heroes  Act  of  201 1 ,9  which 
required  the  Department  of  Labor  to  carry  out  a  demonstration  project 
on  credentialing  for  the  purpose  of  facilitating  the  seamless  transition 
of  members  of  the  armed  forces  from  service  on  active  duty  to  civilian 
employment.10 

•  In  2012,  DOD,  at  the  President’s  direction,  created  the  DOD 
Credentialing  and  Licensing  Task  Force  to  identify  opportunities  for 
servicemembers  to  earn  civilian  occupational  credentials.* 11  The  task 
force  oversees  the  coordination  and  execution  of  DOD  and  military 
service  credentialing  initiatives.  According  to  the  task  force,  they  set 
out  to  (1 )  identify  military  specialties  that  readily  transfer  to  high- 
demand  jobs;  (2)  work  with  key  stakeholders  and  civilian  credentialing 
agencies  to  address  gaps  between  military  training  programs  and 
credentialing  requirements;  and  (3)  provide  servicemembers  with 
greater  access  to  necessary  certification  and  licensing  exams.  The 
task  force’s  first  action  was  to  offer  up  to  126,000  servicemembers  the 
opportunity  to  obtain  machinist,  logistics,  welding,  and  engineering 
certifications  for  high-demand  manufacturing  jobs. 

•  The  NDAA  for  Fiscal  Year  2012  authorized  further  actions  to  support 
servicemembers’  preparation  for  employment  in  the  civilian  sector. 
Section  551  authorized  the  Secretary  of  a  military  department  to  carry 
out  one  or  more  programs  to  provide  eligible  members  of  the  armed 
forces  with  job  and  employment  skill  training  to  help  prepare  them  for 
employment  in  the  civilian  sector.12  Section  558  required  the 


9The  VOW  to  Hire  Heroes  Act  of  2011  also  included  efforts  to  revamp  the  Transition 
Assistance  Program,  which  is  a  gateway  to  information  and  services  available  to 
servicemembers  transitioning  to  civilian  life.  We  previously  reported  on  this  program  in 
Transitioning  Veterans:  Improved  Oversight  Needed  to  Enhance  Implementation  of 
Transition  Assistance  Program,  GAO-14-144  (Washington,  D.C.:  Mar.  5,  2014). 

1038  U.S.C.  §4114,  amended  by  Pub.  L.  No.  112-56,  §  237  (2011). 

11  Military  Skills  for  America’s  Future:  Leveraging  Military  Service  and  Experience  to  Put 
Veterans  and  Military  Spouses  Back  to  Work,  Executive  Office  of  the  President 
(Washington:  D.C.:  May  31, 2012). 

12Pub.  L.  No.  112-81,  §  551  (201 1 )  (codified  as  amended  at  10  U.S.C.  §  1143). 
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Secretary  of  Defense  to  carry  out  a  pilot  program  to  assess  the 
feasibility  of  permitting  servicemembers  to  obtain  civilian  credentialing 
for  skills  required  for  military  occupational  specialties  or  qualification 
for  duty  specialty  codes.13 

•  Section  542  of  the  NDAAfor  Fiscal  Year  2014  required  the 
Secretaries  of  the  military  departments  to  make  information  on  civilian 
credentialing  opportunities  available  to  servicemembers.14 

•  Section  551  of  the  NDAAfor  Fiscal  Year  201 5  amended  10U.S.C.  § 
2015  to  require  DOD  and  the  Department  of  Homeland  Security  (with 
respect  to  the  Coast  Guard)  to  carry  out  a  credentialing  program  that 
allows  servicemembers  to  obtain  professional  credentials  related  to 
military  training  and  skills  acquired  during  service  in  the  armed 
forces.15 

•  Section  559  of  the  NDAAfor  Fiscal  Year  2016  amended  10  U.S.C.  § 
2015  to  add  a  third-party  accreditation  requirement  to  DOD’s 
credentialing  program.16  This  section  requires  that  any  credential  in 
DOD’s  credentialing  program  be  accredited  by  an  accreditation  body 
that  meets  six  requirements  aimed  at  quality  assurance.  The  new 
third-party  accreditation  requirement  must  be  implemented  no  later 
than  3  years  after  the  date  of  the  enactment  of  the  NDAA  for  Fiscal 
Year  2016. 


13Pub.  L.  No.  112-81,  §  558  (2011). 

14Pub.  L.  No.  113-66,  §  542  (2013). 

15Pub.  L.  No.  113-291,  §  551  (2014)  (codified  as  amended  at  10  U.S.C.  §  2015). 
16Pub.  L.  No.  114-92,  §  559  (2015)  (codified  as  amended  at  10  U.S.C.  §  2015). 
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DOD  Has  Taken 
Steps  to  Establish  the 
Statutorily  Required 
Credentialing 
Program  but  Has  Not 
Developed 
Performance 
Measures,  and 
Challenges  Remain 


While  DOD  has  taken  steps  to  establish  the  statutorily  required 
credentialing  program,  it  has  not  developed  and  implemented 
performance  measures  to  gauge  the  effectiveness  of  the  program,  and 
some  challenges  remain.  Although  the  NDAA  for  Fiscal  Year  2015  does 
not  statutorily  require  performance  measures,  it  may  be  difficult  to 
determine  whether  DOD’s  credentialing  program  is  on  track  to  achieve 
desired  results  or,  alternatively,  needs  corrective  actions  without  such 
measures. 


DOD  Has  Taken  Steps  to 
Establish  a  Statutorily 
Mandated  Credentialing 
Program 


DOD  has  taken  steps  to  establish  a  credentialing  program  and  provide 
payment  of  expenses  to  servicemembers  obtaining  credentials,  in 
accordance  with  statutory  requirements  established  in  the  NDAA  for 
Fiscal  Year  2015.  Also,  DOD  has  established  a  process  to  help  ensure 
quality  control  while  establishing  its  credentialing  program.  Section  551  of 
the  NDAA  for  Fiscal  Year  2015  amended  10  U.S.C.  §  2015  to  require 
DOD  to  (1)  carry  out  a  program  that  allows  servicemembers  to  obtain 
professional  credentials  related  to  military  training  and  skills  acquired 
during  service  in  the  armed  forces,  which  help  to  qualify  them  for  civilian 
occupations;  and  (2)  provide  for  the  payment  of  expenses  for 
servicemembers  for  professional  accreditation,  federal  occupational 
licenses,  state-imposed  and  professional  licenses,  professional 
certification,  and  related  expenses.17  It  is  up  to  each  service  to  determine 
how  it  will  pay  for  credentials  in  DOD’s  credentialing  program.  According 
to  service  officials,  each  service  paid  for  credentials  in  fiscal  year  2015 
differently.  For  example,  the  Army  took  funding  for  the  program  out  of 
existing  service  funds,  while  the  Air  Force  used  funding  associated  with  a 
budget  activity  for  voluntary  education. 


17Pub.  L.  No.  113-291,  §  551  (2014)  (codified  as  amended  at  10  U.S.C.  §  2015). 
“Expenses”  refers  to  expenses  for  classroom  instruction,  hands-on  training  (and 
associated  materials),  manuals,  study  guides  and  materials,  text  books,  processing  fees, 
and  test  fees  and  related  fees.  Id.  The  statute  also  required  the  Secretary  of  Defense  and 
the  Secretary  of  Homeland  Security,  with  respect  to  the  Coast  Guard,  to  prescribe 
regulations  to  carry  out  the  credentialing  program.  Id. 
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While  Office  of  the  Secretary  of  Defense  (OSD)  officials  are  still  in  the 
process  of  coordinating  a  draft  DOD  instruction  for  the  program  that  will 
assign  responsibilities  and  prescribe  procedures  for  the  operations  of  the 
program,  each  of  the  services  has  established  Credentialing 
Opportunities  On-Line  (COOL)  websites  to  help  their  servicemembers 
find  information  on  certifications  and  licenses  related  to  their  jobs.  Service 
officials  stated  that  they  have  contracts  with  a  private  company  to  help 
them  design,  maintain,  and  update  their  individual  COOL  websites.  The 
contractor  conducts  research  and  analysis  for  military  and  civilian  jobs. 
Servicemembers  can  use  the  COOL  websites  to  obtain  background 
information  on  credentialing  and  find  detailed  information  on  credentials 
related  to  a  military  occupational  specialty,  credential  requirements, 
potential  gaps  between  military  training  and  civilian  credentialing 
requirements,  and  resources  available  to  fill  those  gaps.  In  order  to 
ensure  the  quality  of  the  credentials  on  the  COOL  websites,  DOD  uses 
three  different  mechanisms.  First,  it  has  designed  the  COOL  websites  to 
show  the  accreditation  status  for  credentials  that  have  been  accredited  by 
a  third-party  organization.  Third-party  organizations  evaluated  these 
credentials  to  determine  whether  they  meet  standards  to  verify  their 
validity.  Second,  DOD  has  the  COOL  websites  show  credentials  that 
qualify  for  funding  under  one  of  the  Gl  Bill  programs,  which  provides 
funding  for  servicemembers  to  take  college  courses.  Lastly,  DOD  uses  a 
credentialing  standards  checklist  (see  table  1)  to  help  ensure  the  quality 
of  credentials  on  the  COOL  websites.  DOD  does  not  allow  a  credential  to 
be  listed  on  the  COOL  websites  unless  it  meets  the  conditions  in  this 
checklist. 


Table  1:  DOD’s  Credentialing  Standards  Checklist  Used  to  Determine  Whether  to  Include  a  Credential  on  Its  Website 

Standard/Requirement 

Met? 

Yes  or  No 

Comments/Documentation 

Provide  link(s)  or  submit 
supporting  documentation 

1 

The  credentialing  body  will  not  require  the  candidate  to  have  completed  training 
that  is  solely  provided  by  the  credentialing  body. 

2 

If  the  credentialing  body  does  provide  an  educational  or  training  program,  it  must 
not  imply  that  it  is  the  only  approved  program  as  a  requirement  for  the  credential. 

3 

If  the  credentialing  body  does  provide  an  educational  or  training  program,  those 
individuals  responsible  for  training  must  not  be  affiliated  with  or  responsible  for 
the  development  of  credentialing  standards  or  the  approval  process. 

4 

The  credential  exam  fee  must  not  be  bundled  with  non-examination-related  costs 
(i.e.  training  materials,  membership  cards,  organizational  magazines,  etc.). 

5 

The  credentialing  body  must  document  the  specific  tasks,  knowledge,  job  duties, 
and/or  skills  that  are  purportedly  being  measured  by  the  assessment  process 
(e.g.  exam,  portfolio  review,  requirements  review)  associated  with  the  exam. 
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Standard/Requirement 

Met?  Comments/Documentation 

Yes  or  No  Provide  link(s)  or  submit 
supporting  documentation 

6 

The  credentialing  body  must  document  the  development  of  the  exam 
assessment,  specifying  psychometric  processes. 

7 

The  content  areas  of  the  assessment  or  exam  must  be  directly  aligned  with 
information  gathered  during  a  job/task  analysis,  role  delineation,  and/or  subject- 
matter  expert  consultation. 

8 

Evidence  regarding  the  validity  and  reliability  of  the  assessment/exam  scores 
must  be  documented. 

9 

The  credentialing  body  must  publish  all  assessment/examination  processes  and 
procedures  required  of  the  candidate. 

10 

The  certification/license  must  be  recognized  and  in  use  by  industry/organizations 
other  than  the  credentialing  body’s  organization. 

11 

The  credentialing  body  must  be  accessible  and  responsive  to  both  the  applicant 
and  certification  holder  (i.e.  Internet,  Phone,  Email) 

12 

Membership  in  an  association  affiliated  with  the  credentialing  body  will  not  be 
required  for  initial  certification/license. 

13 

The  certification/license  must  have  a  time  limit  of  Continuing  Education 
requirements,  with  options  for  recertification  and/or  maintenance  clearly  outlined. 

14 

The  credential(s)  offered  must  not  be  tailored  toward  or  centric  to  military 
servicemembers  or  occupations.  It  must  fulfill  a  need  in  the  civilian  workforce. 

Source:  DOD.  |  GAO-17-133 

While  DOD  has  taken  steps  to  establish  the  statutorily  required 
credentialing  program,  it  has  not  developed  and  implemented 
performance  measures  to  gauge  the  success  of  the  program.  OSD 
officials  developed  a  draft  DOD  instruction  for  the  program  that  will  assign 
responsibilities  and  prescribe  procedures  for  the  operation  of  the  program 
and  payment  of  credentialing  expenses,  but  this  instruction  does  not 
include  performance  measures.  OSD  officials  said  that  they  anticipate 
issuing  the  DOD  instruction  for  the  program  early  in  2017.  Although  the 
NDAA  for  Fiscal  Year  2015  does  not  statutorily  require  performance 
measures,  these  measures  are  important  to  monitor  and  assess  the 
quality  of  the  program’s  performance.  OSD  officials  told  us  that  they  do 
not  have  any  performance  measures  or  goals  in  place  to  measure  the 
success  of  the  credentialing  program.  Standards  for  Internal  Control  in 


DOD  Has  Not  Developed 
and  Implemented 
Performance  Measures  for 
Its  Credentialing  Program 
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the  Federal  Government18  require  the  monitoring  of  a  program’s  activities 
to  enable  managers  to  gauge  a  program’s  effectiveness.  When  we 
applied  this  standard  to  DOD’s  draft  instruction,  we  noted  that  the  draft 
assigns  the  responsibility  for  monitoring,  which  is  in  accordance  with  the 
standards;  however,  the  draft  does  not  contain  any  performance 
measures — a  key  standard  for  internal  control,  and  necessary  for 
evaluating  a  program’s  effectiveness. 

By  using  performance  measures,  conducting  ongoing  monitoring,  and 
reporting  on  progress  in  meeting  desired  outcomes  in  a  transparent 
manner,  key  decision  makers  can  obtain  feedback  for  improving  both 
policy  and  operational  effectiveness.  Through  our  prior  work  on 
performance  measurement,  we  have  identified  several  important 
attributes  of  performance  measures  (see  table  2). 19  Also,  we  have 
previously  reported  that  by  tracking  and  developing  a  performance 
baseline  for  all  performance  measures,  agencies  can  better  evaluate 
whether  they  are  making  progress  and  goals  are  being  achieved.20 


18See  GAO/AIM D-00-21 .3.1  and  GAO-14-704G.  Both  versions  of  the  Standards  for 
Internal  Control  in  the  Federal  Government  include  the  internal  control  monitoring. 

19See  GAO,  Tax  Administration:  IRS  Needs  to  Further  Refine  Its  Tax  Filing  Season 
Performance  Measures,  GAO-03-143  (Washington,  D.C.:  Nov.  22,  2002),  p.  45,  fora 
description  of  how  we  developed  the  attributes  of  effective  performance  goals  and 
measures.  We  considered  key  legislation,  such  as  the  Government  Performance  and 
Results  Act  of  1993  and  other  performance  management  literature.  See  also  GPRA 
Performance  Reports,  GAO/GGD-96-66R,  (Washington,  D.C.:  Feb.  14,  1996);  Missile 
Defense:  Opportunity  to  Refocus  on  Strengthening  Acquisition  Management,  GAO-13-432 
(Washington,  D.C.:  Apr.  26,  2013);  Performance  Measurement  and  Evaluation:  Definitions 
and  Relationships,  GAO-1 1-646SP  (Washington,  D.C.:  May  201 1 );  and  Agency 
Performance  Plans:  Examples  of  Practices  That  Can  Improve  Usefulness  to 
Decisionmakers,  GAO/GGD/AIMD-99-69  (Washington,  D.C.:  Feb.  26,  1999). 

on 

See  GAO,  Defense  Fiealth  Care  Reform:  Additional  Implementation  Details  Would 
Increase  Transparency  of  DOD’s  Plans  and  Enhance  Accountability,  GAO-14-49 
(Washington,  D.C.:  Nov.  6,  201 3),  p.  17  for  a  description  of  why  the  baseline  measure  was 
added  as  an  attribute  of  effective  performance  measures. 
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Table  2:  Attributes  of  Effective  Performance  Measures  Identified  by  GAO 

Attribute 

Definition 

Balance 

A  suite  of  measures  ensures  that  an  organization’s  various  priorities  are  covered. 

Clarity 

Measure  is  clearly  stated,  and  the  name  and  definition  are  consistent  with  the  methodology 
used  to  calculate  it. 

Core  program  activities 

Measures  cover  the  activities  that  an  entity  is  expected  to  perform  to  support  the  intent  of  the 
program. 

Government-wide  priorities 

Each  measure  covers  a  priority  such  as  quality,  timeliness,  and  cost  of  service. 

Limited  overlap 

Measure  should  provide  new  information  beyond  that  provided  by  other  measures. 

Linkage 

Measure  is  aligned  with  division  and  agency-wide  goals  and  mission  and  is  clearly 
communicated  throughout  the  organization. 

Measurable  target 

Measure  has  a  numerical  goal. 

Objectivity 

Measure  is  reasonably  free  from  significant  bias  or  manipulation. 

Reliability 

Measure  produces  the  same  result  under  similar  conditions. 

Baseline  and  trend  data 

Measure  has  a  baseline  and  trend  data  associated  with  it  to  identify,  monitor,  and  report 
changes  in  performance  and  to  help  ensure  that  performance  is  viewed  in  context. 

Source:  GAO.  |  GAO-17-133 

OSD  has  not  provided  any  guidance  for  using  performance  measures  for 
the  program,  but  it  asked  the  services  in  fiscal  year  2015  to  compile  and 
report  quarterly  the  numbers  of  licenses  and  certificates  they  fund  for 
each  military  occupation.  The  Army,  the  Air  Force,  and  the  Navy  reported 
these  data,  but  the  Marine  Corps  did  not,  as  it  did  not  begin  paying  for 
credentials  until  the  end  of  fiscal  year  2015.  Specifically,  OSD  asked  the 
services  to  report  data  by  military  occupation  on  the  number  of 
credentials  that  active  duty  and  reserve  servicemembers  successfully 
attained  (see  table  3)  and  expenditures  for  the  credentialing  program  (see 
table  4). 

Table  3:  Credentials  Attained  by  Servicemembers  in  Fiscal  Year  201 5 

Service 

Credentials  attained 

Army 

13,828 

Air  Force 

78 

Navy 

6,493 

Marine  Corps 

— 

Total 

20,399 

Source:  GAO  analysis  of  service  data.  |  GAO-17-133 

Notes:  Totals  shown  include  only  credentials  obtained  voluntarily  by  servicemembers,  not  the 
numbers  servicemembers  obtained  as  part  of  their  mandatory,  regular  military  training.  The  Marine 
Corps  did  not  have  data  to  report  in  fiscal  year  2015  because  it  only  started  paying  for  credentials  at 
the  end  of  the  year. 


Page  11 


GAO-17-133  Military  Personnel 


Table  4:  Department  of  Defense  Credentialing  Program  Expenditures  in  Fiscal  Year 
2015 

Army 

Air  Force 

Navy 

Total 

Credentialing  exams 

$1,406,670 

$114,832 

$2,183,807 

$3,705,309 

Credentialing  renewals 

— 

— 

23,147 

23,147 

Study  materials 

498,848 

— 

— 

498,848 

Classroom  instruction 

1,076,308 

— 

— 

1,076,308 

Processing  fees 

37,992 

3,702 

36,395 

78,089 

Membership  fees 

179,465 

— 

— 

179,465 

Other 

618,965 

— 

— 

618,965 

Total 

$3,818,248 

$118,534 

$2,243,349 

$6,180,131 

Source:  GAO  analysis  of  service  data.  |  GAO-17-133 

Notes:  Totals  shown  include  only  credentials  obtained  voluntarily  by  servicemembers,  not  the 
numbers  servicemembers  obtained  as  part  of  their  mandatory,  regular  military  training.  The  Army 
numbers  represent  the  Training  and  Doctrine  Command  only  and  do  not  include  any  financial 
information  from  the  Medical  Command.  The  Marine  Corps  did  not  have  data  to  report  in  fiscal  year 
20 1 5  because  it  only  started  paying  for  credentials  at  the  end  of  the  year. 


OSD  officials  collected  program  data  from  the  services  in  fiscal  year 
2015,  but  stated  that  it  is  difficult  to  make  observations  about  the  outlook 
of  the  program  given  that  their  data  pertain  only  to  that  one  year.  Officials 
said  that  they  need  to  obtain  more  data  so  that  they  can  better 
understand  the  effects  of  the  program.  They  said  they  have  modified  the 
fiscal  year  2016  data  call  for  the  services  and  are  hoping  that  the  new 
reporting  format  will  provide  more  useful  data.  OSD  officials  said  that  for 
fiscal  year  2016,  the  services  will  no  longer  report  credentialing  data  by 
military  occupation  but  rather  will  report  these  data  at  a  higher,  aggregate 
level.  For  example,  instead  of  reporting  credentialing  data  for  every  health 
care-related  military  occupation,  the  services  will  report  a  roll-up  of  the 
data  for  all  those  occupations  under  one  category.  OSD  officials  said  that 
collecting  credentialing  data  for  every  military  occupation  resulted  in 
having  too  much  data  that  they  found  not  to  be  useful.  Officials  said  they 
believe  that  consolidation  of  the  data  at  a  higher,  collective  level  will  be 
more  helpful  when  reviewing  the  number  of  credentials  that 
servicemembers  attained  and  expenditures  for  the  program.  OSD  officials 
also  stated  that  they  need  to  develop  performance  measures  and  goals 
but  that  they  were  waiting  until  they  finished  the  draft  DOD  instruction  for 
the  program.  While  we  agree  that  finishing  the  draft  DOD  instruction  for 
the  program  is  important,  we  believe  that  developing  performance 
measures  for  the  program  that  have  a  baseline  by  which  future  progress 
is  to  be  measured  should  be  established  early  in  the  program,  and  that 
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awaiting  the  final  DOD  instruction’s  issuance  may  inhibit  DOD’s  ability  to 
ensure  that  its  program  goals  are  being  met. 

Without  performance  measures  for  the  program,  decision  makers  will  lack 
transparency  over  DOD’s  efforts  to  improve  the  credentialing  of 
servicemembers.  Specifically,  without  performance  measures  that  have 
targets  and  a  baseline  against  which  to  measure  current  performance, 
DOD  officials  and  other  decision  makers,  such  as  members  of  Congress, 
may  find  it  difficult  to  determine  whether  DOD’s  credentialing  program  is 
on  track  to  achieve  desired  results  or,  alternatively,  needs  corrective 
actions. 


DOD  Has  Identified 
Challenges  in  Meeting 
Statutory  Requirements 
for  the  Credentialing 
Program 


After  the  NDAA  for  Fiscal  Year  2016  was  passed,  OSD  officials 
expressed  concern  over  the  challenge  posed  by  the  new  third-party 
accreditation  requirement  placed  on  DOD’s  credentialing  program. 
Specifically,  section  559  of  the  NDAA  for  Fiscal  Year  201 6  amended  10 
U.S.C.  §  2015  to  require  that  any  credential  in  DOD’s  credentialing 
program  be  accredited  by  an  accreditation  body  that  meets  six 
requirements21  aimed  at  quality  assurance.  The  new  third-party 
accreditation  requirement  must  be  implemented  no  later  than  January 
2018. 


OSD  officials  said  that  about  70  percent  of  the  credentials  in  the  program 
and  listed  on  the  COOL  website  are  not  accredited  by  a  third-party 
agency.  OSD  officials  said  that,  even  though  some  credentials  are  not 
accredited  by  a  third-party,  they  are  still  widely  accepted  within  the  private 
sector  and  are  considered  the  industry  standard.  OSD  officials  also  said 


21  Title  10,  section  2015  of  the  U.S.  Code,  amended  by  Pub.  L.  No.  114-92,  §  559, 
requires  that  the  accreditation  body:  (1)  be  an  independent  body  that  has  in  place 
mechanisms  to  ensure  objectivity  and  impartiality  in  its  accreditation  activities;  (2)  meet  a 
recognized  national  or  international  standard  that  directs  its  policy  and  procedures 
regarding  accreditation;  (3)  apply  a  recognized  national  or  international  certification 
standard  in  making  its  accreditation  decisions  regarding  certification  bodies  and  programs; 
(4)  conduct  on-site  visits,  as  applicable,  to  verify  the  documents  and  records  submitted  by 
credentialing  bodies  for  accreditation;  (5)  have  in  place  policies  and  procedures  to  ensure 
due  process  when  addressing  complaints  and  appeals  regarding  its  accreditation 
activities;  and  (6)  conduct  regular  training  to  ensure  consistent  and  reliable  decisions 
among  reviewers  conducting  accreditations.  Additionally,  the  accreditation  body  must 
meet  such  other  criteria  as  the  Secretary  concerned  considers  appropriate  in  order  to 
ensure  quality  in  its  accreditation  activities. 
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that  DOD  uses  a  quality  assurance  process  to  ensure  that  the  credentials 
available  to  servicemembers  are  of  high  quality.  While  OSD  officials 
identified  the  third-party  accreditation  requirement  as  a  challenge  to 
DOD’s  credentialing  program,  subsequent  congressional  proposals  would 
give  DOD  alternatives  for  ensuring  quality  over  its  program.  Specifically, 
section  561  of  H.R.  4909  and  section  562  of  S.  2943  for  the  NDAA  for 
Fiscal  Year  2017  contain  a  proposed  amendment  that  would  provide 
alternatives  to  the  third-party  accreditation  requirement.  The  proposed 
amendment  states  that  any  credential  in  DOD’s  credentialing  program 
would  need  to  meet  one  of  the  following  four  requirements: 

•  be  accredited  by  a  nationally  recognized  third-party  personnel 
certification  program  accreditor; 

•  be  sought  or  accepted  by  employers  within  the  industry  or  sector 
involved  as  a  recognized,  preferred,  or  required  credential  for 
recruitment,  screening,  hiring,  retention,  or  advancement  purposes; 
and,  where  appropriate,  be  endorsed  by  a  nationally  recognized  trade 
association  or  organization  representing  a  significant  part  of  the 
industry  or  sector; 

•  grant  licenses  that  are  recognized  by  the  federal  government  or  a 
state  government;  or 

•  meet  credential  standards  of  a  federal  agency. 

OSD  officials  said  that  the  proposed  amendment  would  eliminate  the 
challenge  caused  by  the  statute’s  third-party  accreditation  requirement 
and  would  give  them  more  flexibility  in  identifying  and  paying  for  valid 
credentials  for  servicemembers. 


DOD  Engages  with 
States  to  Further 
Assist 

Servicemembers  in 
Attaining  Credentials 


DOD  engages  with  states  in  various  efforts  to  further  assist 
servicemembers  in  attaining  credentials.  For  example,  DOD  works  at  the 
state  level  through  its  Defense  State  Liaison  Office  to  encourage  and 
support  the  implementation  of  laws  and  regulations  that  assist 
servicemembers  in  obtaining  credentials  aimed  at  improving  their 
employability.  These  efforts  vary  by  state.  DOD  also  worked  with  the 
Department  of  Labor  and  the  National  Governors  Association’s  Center  for 
Best  Practices  to  implement  a  statutorily  required  demonstration  project  in 
six  states  to  identify  and  address  barriers  to  servicemembers’  ability  to 
obtain  credentials. 
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DOD  Works  with  States  to 
Support  the  Development 
of  Servicemember 
Credentialing  Efforts 


DOD  works  with  states  to  advance  the  development  of  credentialing 
initiatives  for  servicemembers,  but  the  specific  efforts  and  results  vary 
from  state  to  state.  To  address  this,  DOD  created  the  “USA  4  Military 
Families”  initiative,  which  is  managed  by  DOD’s  Defense  State  Liaison 
Office  and  focuses  on  engaging  with  state-level  policymakers,  not-for- 
profit  associations,  concerned  business  interests,  and  other  state  leaders 
regarding  the  needs  of  military  members  and  their  families.  In  developing 
these  networks  and  partnerships,  DOD  seeks  to  work  at  the  state  level  to 
remove  unnecessary  barriers  and  significantly  improve  the  quality  of  life 
for  military  families. 

To  identify  the  issues  on  which  the  USA  4  Military  Families  initiative  will 
focus,  DOD  reviews  state-related  issues  annually.  The  Defense  State 
Liaison  Office  requests  input  on  potential  issues  from  external 
organizations,  personnel  and  readiness  organizations,  military  services, 
the  joint  staff,  and  the  National  Guard.  The  input  is  analyzed,  the  DOD 
offices  that  provided  input  to  create  the  list  are  consulted  again,  and  the 
Defense  State  Liaison  Office  develops  a  list  of  priorities.  The  final  list  of 
priorities  is  approved  by  the  Assistant  Secretary  of  Defense  for  Military 
and  Reserve  Affairs  (Readiness  and  Force  Management).  One  of  the 
2016  issue  areas  that  the  initiative  has  identified  and  focused  on  is  the 
development  of  credentialing  programs  at  the  state  level.  The  Defense 
State  Liaison  Office  has  encouraged  state  legislators  to  establish 
requirements  for  licensure  boards  through  state-level  statutes  and 
regulations  to  do  the  following: 

•  allow  separating  servicemembers  to  obtain  credit  toward  occupational 
licenses  based  on  their  military  education,  training,  and  experience 
that  is  essentially  equivalent  to  licensing  requirements; 

•  allow  separating  servicemembers  remaining  in  that  state  to  transfer  a 
current  license  from  another  state  through  endorsement  or  temporary 
licensing; 

•  allow  deactivating  reserve  members  to  practice  in  the  state  of  origin 
for  a  temporary  period  with  a  license  that  would  have  otherwise 
expired  while  on  active  duty;  and 

•  establish  requirements  for  academic  institutions  to  grant  separating 
servicemembers  credit  toward  degree  and  certificate  requirements  for 
education,  training,  and  experience  gained  in  the  military. 

According  to  DOD  officials,  between  the  years  2013  and  2015  state 
legislators  in  37  states  enacted  66  bills  related  to  military  servicemember 
credentialing  and  credit  for  military  training,  education,  and  experience. 
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For  example,  in  2014  Minnesota  enacted  a  bill  that  makes  expedited  and 
temporary  licensing  available  to  veterans  in  health  care  professions  such 
as  optometry,  marriage  and  family  therapy,  clinical  counseling,  and 
podiatry.  In  2015  and  2016  several  states,  including  Texas  and  Colorado, 
adopted  interstate  agreements  to  acknowledge  licensed  emergency 
medical  service  personnel  from  other  member  states,  giving  those 
professionals  the  ability  to  work  in  any  of  the  member  states. 

We  met  with  officials  from  DOD’s  Defense  State  Liaison  Office  and  the 
department’s  South  Atlantic  and  Southeast  regional  liaison  offices,  as  well 
as  with  state-level  government  officials  in  three  states  and  an  official 
representing  a  non-government  state-level  organization,  to  obtain  their 
views  on  DOD  efforts  to  work  at  the  state  level  to  assist  servicemembers 
in  attaining  credentials.  The  DOD  officials  stated  that  they  work  to 
educate  and  inform  legislators  and  policymakers  about  the  issues  in 
which  DOD  is  most  interested.  Officials  we  met  with  in  Georgia,  Florida, 
and  Virginia  told  us  that  individual  states  focus  on  those  employment 
issues  that  reflect  their  state  economies.  According  to  officials  with  the 
state  of  Florida’s  Department  of  Business  and  Professional  Regulation, 
Florida  focuses  on  the  employment  of  servicemembers  once  they  have 
left  military  service,  as  opposed  to  working  to  attract  servicemembers  to 
the  state  based  on  the  skills  and  competencies  they  acquired  while  in  the 
military.  According  to  Florida  state  officials,  the  state’s  three  largest 
credentialing  programs  are  real  estate,  cosmetology,  and  construction. 
Two  of  those  programs,  real  estate  and  cosmetology,  are  not  aligned  with 
any  military  service  occupations.  Officials  with  the  Commonwealth  of 
Virginia’s  Secretary  of  Veterans  and  Defense  Affairs  told  us  that  they 
focus  on  attracting  servicemembers  who  possess  training,  education, 
skills,  or  knowledge  gained  while  in  military  service  that  are  in  line  with 
occupations  typically  found  within  the  state  of  Virginia.  For  example, 
Virginia  is  working  to  grow  and  develop  manufacturing  businesses  within 
the  state,  so  it  seeks  to  attract  servicemembers  who  have  the  necessary 
skills  and  abilities  to  contribute  to  that  industry.  In  Georgia,  an  official  with 
the  Camden  Partnership — a  nonprofit  organization  that  promotes 
community  support  for  Naval  Submarine  Base  Kings  Bay — told  us  that 
that  organization  works  directly  with  the  community  and  the  installation, 
and  not  with  DOD,  to  encourage  retiring  and  separating  servicemembers 
to  remain  in  the  community.  For  example,  the  nonprofit  organization 
worked  with  the  naval  installation  to  identify  and  align  military  occupations 
assigned  to  the  installation  with  civilian  occupations  that  served  the  local 
community. 
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Additionally,  Virginia  passed  a  law  that  requires  the  Virginia  Board  of 
Education  to  provide  a  provisional  teaching  license,  valid  for  a  period  not 
to  exceed  3  years,  to  any  former  member  of  the  armed  forces  of  the 
United  States  or  the  Virginia  National  Guard  who  has  received  an 
honorable  discharge  and  has  the  appropriate  level  of  experience  or 
training  but  does  not  meet  the  requirements  for  a  renewable  teacher 
license.22  Georgia  passed  a  law  to  require  professional  licensing  boards 
and  other  boards  to  adopt  rules  and  regulations  for  implementing  a 
process  by  which  transitioning  servicemembers  may  qualify  for  temporary 
licenses,  licenses  by  endorsement,  expedited  licenses,  or  a  combination 
thereof.23 


22Va.  Code  Ann.  §  22.1-298.1. 
23Ga.  Code  Ann.  §43-1-34. 
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DOD  Assisted  the 
Department  of  Labor 
and  the  National 
Governors 
Association  in 
Implementing  a 
Demonstration 
Project  in  Six  States 
to  Assist 

Servicemembers  in 
Attaining  Credentials 


DOD  provided  data  and  subject  matter  experts  to  the  Department  of 
Labor  and  the  National  Governors  Association  to  implement  a 
demonstration  project  to  identify  state-level  professional  requirements 
that  are  met  through  military  training,  along  with  strategies  to  remove 
barriers  to  relevant  credentials  and  license.  Section  41 14  of  Title  38  of  the 
U.S.  Code,  as  amended  by  section  237  of  the  VOW  to  Hire  Heroes  Act  of 
2011,  required  a  demonstration  project  on  credentialing  for  the  purpose  of 
facilitating  the  seamless  transition  of  servicemembers  from  service  on 
active  duty  to  civilian  employment.24  The  statute  directed  the  Department 
of  Labor  to  identify  federal,  state,  and  industry  credentialing  requirements 
that  are  met  through  the  training  that  servicemembers  receive  while 
serving  in  the  armed  forces,  as  well  as  to  identify  strategies  to  remove 
barriers  to  servicemembers’  efforts  to  attain  credentials.  In  response,  the 
Department  of  Labor  contracted  with  the  National  Governors  Association 
to  carry  out  an  18-month  demonstration  project  designed  to  meet  those 
requirements.25 

DOD  supported  the  demonstration  project  by  providing  the  National 
Governors  Association  with  data  and  subject  matter  experts  to  assist  in 
project  development.  Specifically,  DOD’s  Defense  Manpower  Data  Center 
provided  information  on  the  distribution  of  recently  separated 
servicemembers  associated  with  each  of  the  states  chosen  to  participate 
in  the  project.  DOD  also  provided  subject  matter  experts  who  reviewed 
and  commented  on  documents  used  to  provide  guidance  to  the  states 
throughout  the  demonstration  project,  as  well  as  reviewed  and 
commented  on  the  National  Governors  Association’s  final  report. 


Using  DOD  data,  the  National  Governors  Association  selected  six 
states — Illinois,  Iowa,  Minnesota,  Nevada,  Virginia,  and  Wisconsin — to 
participate  in  the  18-month  project.  Each  state  selected  up  to  three  high- 


2438  U.S.C.  §41 14(a),  amended  by  VOW  to  Hire  Heroes  Act  of  2011,  Pub.  L.  No.  112-56, 
§  237(a)  (201 1 ).  The  statute  authorized  the  demonstration  project  to  be  carried  out  for  a 
period  of  2  years  beginning  on  the  date  of  enactment  of  the  VOW  to  Hire  Heroes  Act  of 
2011.  Id.  at  §41 14(d),  amended  by  Pub.  L.  No.  112-56,  §  237(a)(3)  (2011). 

25See  National  Governors  Association  Center  for  Best  Practices,  Workforce  Development 
Program,  Veterans’  Licensing  and  Certification  Demonstration:  A  Summary  of  State 
Experiences ,  Preliminary  Findings ,  and  Cost  Estimates  (Sept.  2015).  GAO  did  not  review 
or  validate  the  methods  or  results  of  this  project. 
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demand  occupations  that  correspond  with  one  of  three  pre-selected 
military  occupational  specialties:  medic,  police,  and  truck  driver.26 

According  to  the  National  Governor’s  Association,  the  six  states 
participating  in  the  project  found  that  transitioning  servicemembers  and 
veterans  encounter  the  following  three  types  of  barriers  when  trying  to 
attain  civilian  credentials: 

•  Veterans  who  have  military  training  and  experience  that  are 
equivalent  to  the  training  and  experience  of  licensed  civilians  often 
find  that  civilian  licensing  boards  are  not  accustomed  to  recognizing 
the  military  documentation  of  their  training  and  experience. 

•  Veterans  who  experience  gaps  between  their  military  training  and 
experience  and  civilian  requirements  may  have  to  participate  in 
duplicative  training  to  attain  relevant  licensure  or  certification. 

•  Administrative  rules  and  processes  within  civilian  credentialing 
systems  may  create  hurdles  for  veterans  in  obtaining  licensure  or 
certification  that  are  unrelated  to  their  ability  to  competently  provide 
professional  services  to  the  public. 

The  project  also  identified  strategies,  such  as  the  following,  for  states  to 
accelerate  the  licensing  and  certification  of  veterans  based  on  the  barriers 
identified: 

•  To  address  equivalency  challenges,  states  can  assess  the 
equivalency  of  military  training  courses  and  use  official  documentation 
to  permit  veterans  with  fully  or  partially  equivalent  training  and 
experience  to  sit  for  civilian  licensure  examinations,  or  to  license 
veterans  by  endorsement. 

•  To  address  training  gaps,  states  can  work  with  educational  institutions 
to  set  up  accelerated  programs  for  veterans  that  bridge  gaps,  provide 
veterans  with  advanced  standing  in  existing  programs,  or  offer  bridge 
courses  that  prepare  veterans  to  enter  existing  programs. 


2638  USC  §  41 14(b)(1)  required  the  Assistant  Secretary  for  Veterans’  Employment  and 
Training,  in  consultation  with  the  Assistant  Secretary  for  Employment  and  Training,  to 
select  not  more  than  five  military  occupational  specialties  for  purposes  of  the 
demonstration  project.  Each  specialty  selected  by  the  Assistant  Secretary  for  Veterans’ 
Employment  and  Training  shall  require  a  skill  or  set  of  skills  that  is  required  for  civilian 
employment  in  an  industry  with  high  growth  or  high  worker  demand.  Id. 
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•  To  address  administrative  or  processing  challenges,  states  can 
assess  any  non-skill-related  requirements  that  might  disadvantage 
veterans,  such  as  fees  or  length  of  experience,  or  can  take  steps  to 
make  civilian  employment  pathways  friendlier  to  veterans  through 
concerted  outreach  to  both  veterans  and  prospective  employers. 

In  its  final  report,  the  National  Governors  Association  Center  for  Best 
Practices  stated  that  five  of  the  six  states  included  in  the  project  have 
implemented  policies  to  allow  veterans  to  apply  for  licensure  reciprocity, 
and  three  of  the  states  have  implemented  protocols  to  allow  veteran 
licensure  by  examination.  All  six  states  have  engaged  in  the  development 
of  bridge  programs,  three  of  which  were  operational  by  the  close  of  the 
projects’  demonstration  periods.  In  addition,  five  states  have  established 
strategies  to  grant  advanced  standing  to  veterans  in  training  programs, 
and  three  states  have  implemented  strategies  to  streamline  administrative 
rules  and  processes. 


Conclusions 


DOD  has  made  progress  in  its  efforts  to  support  servicemembers  who  are 
transitioning  from  the  armed  forces  into  the  civilian  workforce  by 
establishing  the  statutorily  mandated  credentialing  program  and  engaging 
with  the  states.  Key  to  these  efforts  are  programs  that  translate  the  skills 
that  servicemembers  have  learned  while  serving  in  the  armed  forces  to 
feasibility  for  jobs  in  the  civilian  workforce.  However,  DOD  has  not 
established  measures  for  the  program’s  performance  and  the  extent  to 
which  it  is  achieving  desired  goals  and  outcomes.  Without  measures  that 
go  beyond  simply  counting  the  numbers  of  credentials  obtained  by 
servicemembers,  DOD  will  be  unable  to  fully  assess  its  efforts  to  assist 
servicemembers  in  attaining  credentials,  and  it  will  be  difficult  for  the 
department  to  reliably  determine  whether  its  credentialing  program  is  both 
meeting  desired  goals  and  outcomes  and  improving  servicemembers’ 
ability  to  successfully  enter  the  civilian  workforce. 


Recommendation 
Executive  Action 


To  improve  the  management  of  DOD’s  credentialing  program  and  better 
determine  whether  the  program  is  achieving  its  desired  results,  we 
recommend  that  the  Secretary  of  Defense  direct  the  Under  Secretary  of 
Defense  for  Personnel  and  Readiness  to  develop  and  implement  program 
performance  measures  that  include  key  attributes,  such  as  a  baseline  and 
goals,  that  can  be  used  to  assess  performance. 
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Agency  Comments 
and  Our  Evaluation 


We  provided  a  draft  of  this  report  to  DOD  and  the  Department  of  Labor  for 
review  and  comment.  In  its  comments  on  a  draft  of  this  report,  DOD  did 
not  concur  with  our  recommendation.  DOD’s  comments  are  reprinted  in 
their  entirety  in  appendix  II.  DOD  provided  technical  comments,  which  we 
incorporated  into  the  report  as  appropriate.  The  Department  of  Labor 
provided  technical  comments  on  a  draft  of  this  report,  which  we 
incorporated  into  the  report  as  appropriate. 

DOD  did  not  concur  with  our  recommendation  to  develop  and  implement 
performance  measures  for  its  credentialing  program  that  include  key 
attributes,  such  as  a  baseline  and  goals  that  can  be  used  to  assess 
performance.  In  its  written  comments,  DOD  stated  that  the  services 
neither  require  nor  force  servicemembers  to  earn  credentials,  which  are 
not  required  by  their  military  duty  or  position.  DOD  further  stated  that 
establishing  any  criteria  that  might  create  a  perverse  incentive  to  force 
servicemembers  into  earning  voluntary  credentials  would  be 
counterproductive,  and  servicemembers  would  feel  pressured  to  earn 
credentials  they  may  neither  need  nor  want.  We  do  not  believe  that 
establishing  a  performance  measure  that  entails  tracking  the  numbers  of 
credentials  earned  over  time  and  comparing  these  numbers  to  a  baseline 
would  create  a  perverse  incentive  for  servicemembers.  Instead,  we  are 
suggesting  that  for  management  purposes,  DOD  would  find  the  tracking 
of  how  many  servicemembers  are  obtaining  these  credentials  useful  in 
determining  servicemember  awareness  of  the  program,  how  to  reach 
greater  numbers  of  servicemembers,  and  increase  the  usefulness  of  its 
credentialing  program. 

DOD  also  stated  that  it  has  put  in  place  a  basic  reporting  system  to 
capture  credential  attainment  and  associated  costs  that  provide  the 
necessary  information  to  gauge  program  performance  and  that  it 
supplements  this  information  by  sponsoring  periodic,  point-in-time  studies 
to  assess  the  credentialing  program.  DOD  explained  that  it  believes  this 
core  performance  information  and  the  periodic  studies  will  provide  the 
necessary  information  to  assess  the  efficacy  of  established  credentialing 
programs  and  will  allow  it  to  make  adaptations,  as  necessary.  DOD’s 
efforts  to  capture  credential  attainment  and  associated  cost  information  is 
a  positive  step  to  help  gauge  program  performance  and  will  be  supported 
by  DOD’s  plans  to  supplement  this  information  through  periodic  studies. 
However,  capturing  data  and  conducting  periodic,  point-in-time  studies  to 
assess  a  program  does  not  constitute  measuring  an  agency’s  progress 
against  a  goal.  As  we  note  in  our  report,  by  tracking  and  developing  a 
performance  baseline  for  all  performance  measures,  agencies  can  better 
evaluate  whether  they  are  making  progress  and  goals  are  being 
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achieved.  As  we  conducted  our  work,  and  as  we  note  in  our  report,  DOD 
officials  responsible  for  this  program  told  us  that  they  agreed  that  they 
needed  to  develop  performance  measures  and  goals  but  that  they  were 
waiting  until  they  finished  the  draft  DOD  instruction  for  the  program.  We 
continue  to  believe  that  developing  performance  measures  for  the 
program  that  have  a  baseline  by  which  future  progress  is  to  be  measured 
should  be  established  early  in  the  program  and  that  awaiting  the  final 
DOD  instruction’s  issuance  may  inhibit  DOD’s  ability  to  ensure  that  its 
program  goals  are  being  met. 

In  its  response,  DOD  also  noted  that  the  numbers  of  credentials  we 
include  in  two  tables  of  our  draft  report  are  limited  only  to  those  voluntarily 
attained  by  servicemembers,  paid  for  through  the  services’  voluntary 
credentialing  program,  and  do  not  contain  those  awarded  as  part  of 
DOD’s  regular  training.  Including  only  voluntary  credentials  in  those 
tables  was  our  intent  because  the  scope  of  our  work  focused  on  efforts 
being  made  by  DOD  to  help  servicemembers  who  are  seeking  help 
beyond  their  regular,  mandatory  military  training.  We  have  added  notes  to 
tables  3  and  4  of  our  report  to  clarify  what  we  included  in  our  totals. 


We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees;  the  Secretary  of  Defense;  the  Secretary  of  Labor;  the 
Secretaries  of  the  Army,  the  Navy,  and  the  Air  Force;  the  Commandant  of 
the  Marine  Corps;  and  the  Under  Secretary  of  Defense  for  Personnel  and 
Readiness.  In  addition,  the  report  is  available  at  no  charge  on  the  GAO 
website  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  about  this  report,  please  contact 
me  at  (202)  512-3604  or  farrellb@gao.gov.  Contact  points  for  our  Offices 
of  Congressional  Relations  and  Public  Affairs  may  be  found  on  the  last 
page  of  this  report.  GAO  staff  members  who  made  key  contributions  to 
this  report  are  listed  in  appendix  II. 


Brenda  S.  Farrell 
Director 

Defense  Capabilities  and  Management 
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Appendix  I:  Comments  from  the  Department 
of  Defense 


Subsequent  to  the 
Department  of 
Defense’s  letter,  the 
GAO  report  number 
was  revised  from 
GAO-16-838  to  the 
current  report  number 
GAO-17-133. 


OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 

4000  DEFENSE  PENTAGON 
WASHINGTON,  D.C.  20301-4000 


READINESS 

SEP  2  7  2016 

Ms.  Brenda  S.  Ferrell 

Director,  Defense  Capabilities  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  NW 
Washington  DC  20548 

Dear  Ms.  Ferrell: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  Draft  Report  1 6-838, 
“MILITARY  PERSONNEL:  Performance  Measures  Needed  to  Determine  How  Well  DoD's 
Credentialing  Program  Helps  Service  members,”  dated  August  29,  2016  (GAO  Code  100342). 


Attached  is  DoD’s  response  to  the  one  recommendation  in  the  subject  report.  My  point 
of  contact  is  Frank  C.  DiGiovanni,  who  can  be  reached  at  frank.c.digiovanni.civ@mail.mil,  or 


703-695-2618. 


Sincerely, 


Diana  C.  Banks 

Deputy  Assistant  Secretary  of  Defense 
(Force  Education  and  Training) 


Attachments: 
As  stated 
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Appendix  I:  Comments  from  the  Department  of 
Defense 


Subsequent  to  the 
Department  of 
Defense’s  letter,  the 
table  numbers  were 
revised  from  tables  2 
and  3  to  tables  3  and  4. 


GAO  DRAFT  REPORT  DATED  AUGUST  29,  2016 
16-838  (GAO  CODE  100342) 

“MILITARY  PERSONNEL:  PEFORMANCE  MEASURE  NEEDED  TO  DETERMINE 
HOW  WELL  DOD'S  CREDENTIALING  PROGRAM  HELPS  SERVICEMEMBERS” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATION 


RECOMMENDATION  1:  To  improve  the  management  of  DoD’s  credentialing  program  and 
better  determine  whether  the  program  is  achieving  its  desired  results,  we  recommend  that  the 
Secretary  of  Defense  direct  the  Under  Secretary  of  Defense  for  Personnel  and  Readiness  to 
develop  and  implement  program  performance  measures  that  include  key  attributes,  such  a 
baseline  and  goals,  that  can  be  used  to  assess  performance. 

DoD  RESPONSE:  Non-concur.  More  than  half  of  credentials  earned  by  Service  members  are 
voluntary  in  nature  and  are  completely  at  the  discretion  of  the  Service  member  to  decide  which 
credential  to  pursue  and  when  to  pursue  it.  Therefore,  Services  neither  require  nor  force  Service 
members  to  earn  credentials  which  are  not  required  by  their  military  duty  or  position. 
Establishing  any  criteria  which  might  create  a  perverse  incentive  to  force  Service  members  into 
earning  voluntary  credentials  would  be  counterproductive.  Service  members  would  feel 
pressured  to  earn  credentials  they  may  neither  need  nor  want. 

The  Department  executes  a  credentialing  program  for  three  diverse  reasons:  ( 1 )  Meet  Mandatory 
Occupational  Requirements  -  some  credentials  are  mandated  because  law,  regulation,  or  policy 
requires  a  Service  member  to  have  a  certain  credential  in  order  to  serve  in  a  military  specialty,  or 
designated  capacity.  (2)  Promote  Professionalization  of  the  Force.  Credentials  enhance  the 
professionalism  of  the  Force  by  allowing  Service  members  to  acquire  skills  above  minimum 
occupational  requirements.  These  credentials  are  voluntary.  (3)  Improve  Transition  to  the 
Civilian  Workforce.  Credentials  demonstrate  to  civilian  employers  that  military  training  and 
experience  are  on  par  with  more  traditional  civilian  workforce  training,  education,  and 
experience.  These  credentials  are  also  voluntary. 

The  Department  has  put  in  place  a  basic  reporting  system  to  capture  credential  attainment  and 
associated  costs  that  we  believe  provides  the  necessary  information  to  gauge  program 
performance.  We  supplement  this  information  by  sponsoring  periodic,  point  in  time  studies  to 
assess  our  credentialing  program,  such  as  the  credentialing  study  the  RAND  Corporation 
currently  has  underway  and  the  study  that  the  Center  for  Naval  Analysis  has  recently  completed. 
We  must  also  consider  if  the  data  is  collectible  and  readily  available  within  our  existing 
databases,  available  via  the  good-will  of  other  government  agency  partners,  and  does  not  violate 
Personally  Identifiable  Information  principles.  We  believe  this  core  performance  information 
and  the  periodic  studies  will  provide  the  necessary  information  to  assess  the  efficacy  of 
established  credentialing  programs  and  will  allow  us  to  make  adaptations,  as  necessary. 

It  is  important  to  note  that  the  numbers  of  credentials  included  in  Tables  2  and  3  of  the  GAO 
report  are  limited  only  to  certifications  and  licenses  attained  by  Service  members  and  paid  for 
through  the  Services’  voluntary  credentialing  programs.  They  do  not  include  the  many 
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Appendix  I:  Comments  from  the  Department  of 
Defense 


2 

credentials  Service  members  attain  as  a  requirement  of  their  military  occupations  -  that  is, 
mandatory  credentials;  nor  do  they  include  additional  credentials  awarded  outside  the  Services’ 
voluntary  credentialing  programs,  such  as  those  awarded  by  technical  schools,  apprenticeship 
programs,  and  Community  College  of  the  Air  Force. 
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Appendix  II:  GAO  Contact  and  Staff 
Acknowledgments 


GAO  Contact 


Brenda  S.  Farrell,  (202)  512-3604  orfarrellb@gao.gov 


Staff 

Acknowledgments 


In  addition  to  the  contact  named  above,  key  contributors  to  this  report 
were  Beverly  Schladt,  Assistant  Director;  Stephanie  Moriarty,  Shahrzad 
Nikoo,  Brian  Pegram,  Andrew  Sherrill,  Michael  Silver,  John  Van  Schaik, 
and  Cheryl  Weissman. 


(100342) 


Page  27 


GAO-17-133  Military  Personnel 


GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and  investigative 
arm  of  Congress,  exists  to  support  Congress  in  meeting  its  constitutional 
responsibilities  and  to  help  improve  the  performance  and  accountability  of  the 
federal  government  for  the  American  people.  GAO  examines  the  use  of  public 
funds;  evaluates  federal  programs  and  policies;  and  provides  analyses, 
recommendations,  and  other  assistance  to  help  Congress  make  informed 
oversight,  policy,  and  funding  decisions.  GAO’s  commitment  to  good  government 
is  reflected  in  its  core  values  of  accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost  is 
through  GAO’s  website  (http://www.gao.gov).  Each  weekday  afternoon,  GAO 
posts  on  its  website  newly  released  reports,  testimony,  and  correspondence.  To 
have  GAO  e-mail  you  a  list  of  newly  posted  products,  go  to  http://www.gao.gov 
and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of  production  and 
distribution  and  depends  on  the  number  of  pages  in  the  publication  and  whether 
the  publication  is  printed  in  color  or  black  and  white.  Pricing  and  ordering 
information  is  posted  on  GAO’s  website,  http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card,  MasterCard, 

Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates.  Listen  to  our  Podcasts. 

Visit  GAO  on  the  web  at  www.gao.gov. 

To  Report  Fraud, 

Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512-4400, 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7125, 
Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 

Washington,  DC  20548 

Strategic  Planning  and 
External  Liaison 

James-Christian  Blockwood,  Managing  Director,  spel@gao.gov,  (202)  512-4707 
U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7814, 
Washington,  DC  20548 
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